The present invention relates to data structures for image transmission, methods of image transmission, image decoding apparatus, and data recording media. More particularly, the invention relates to a data structure of a coded image signal which corresponds to each of objects constituting a moving picture and includes locational information of data relating to the object, a method of transforming an image signal of each object into a coded image signal having the data structure and transmitting the image signal, an apparatus for decoding a coded image signal of the data structure, and a data recording medium which contains a program for performing the image transmission method or a program for performing image processing by the image decoding apparatus.
In order to store or transmit digital image information with high efficiency, it is necessary to compressively encode the digital image information. As a typical method of compressive coding of digital image information, there is discrete cosine transformation (DCT) represented by JPEG (Joint Photographic Experts Group) and MPEG (Moving Picture Experts Group). Besides the DCT, there are waveform coding methods such as sub-band coding, wavelet coding, and fractal coding.
Further, in order to eliminate redundant image information between images, such as adjacent frames, inter-frame prediction using motion compensation is carried out. In this method, a pixel value of a pixel in the present frame is represented by a difference between this pixel value and a pixel value of a pixel in the previous frame, and this difference signal is subjected to waveform coding.
A method or compressively coding and transmitting an image signal in objects units has recently been put to practical use. To be specific, plural objects constituting a frame are individually subjected to compressive coded and transmitted so that reproduction of the image signal can be carried out in regions corresponding to the respective objects.
In this method, on the reproduction end, coded image signals corresponding to the respective objects are decoded, and the decoded images of the respective objects are synthesized to display an image corresponding to one frame. This object-by-object coding enables the user to freely combine images of objects to be displayed, whereby edition of a moving picture is facilitated.
Furthermore, in the above-mentioned, it is possible to display a moving picture without reproducing images of relatively unimportant objects, according to the busyness of transmission line, the performance of reproduction apparatus, and the tastes of viewer.
Moreover, in the above-mentioned method, scalability in objects units, i.e., a hierarchical processing, is realized. In the hierarchical processing, image signals corresponding to plural hierarchies and having different resolutions are used as image signals corresponding to the respective objects, and coding or decoding of an image signal of each hierarchy is carried out using an image signal of another hierarchy.
In recent years, an image supply system that enables the user to access moving pictures through computer network, has been spread.
FIG. 9 is a schematic diagram for explaining such an image supply system.
In an image supply system 700 shown in FIG. 9, a plurality of networks 701, 702 and 703 are connected to each other, and a plurality of servers 705, 706 and 707 for supplying image information are connected to the networks 701, 702 and 703, respectively. Further, a plurality of terminal units for receiving and reproducing images (hereinafter referred to simply as xe2x80x9cterminal unitsxe2x80x9d) are connected to the respective networks 701xcx9c703. To simplify the description, only a terminal unit 704 connected to the network 701 is shown in FIG. 9.
In the image supply system so constructed, when the terminal unit 704 is supplied with image information from the server 705, initially, the terminal unit 704 contacts the server 705 and forms a line connecting the terminal unit 704 with the server 705. Thereafter, the terminal unit 704 receives image data of a predetermined program supplied from the server 705, and reproduces the image data by decoding.
Further, in order to make the terminal unit 704 able to receive data relating to the received image data, i.e., the program (hereinafter referred to as relevant data), locational information showing resource locations of the relevant data is described in a text file corresponding to an image file in which the image data is described. The text file and the image file have different formats. Accordingly, the user can obtain the relevant data of the program received by the terminal unit according to the locational information described in the text file.
For example, Internet, being an image supply system as mentioned above, employs Hypertext files. Described in the text files are character groups and locational information showing resource locations of other data relating to the character groups (relevant data). When the user selects a specific character group, data relating to the selected character group is supplied from a prescribed server. That is, these character groups are linked to other documents, such as text file, program file, and graphic file.
In the current Internet, with respect to moving pictures, locational information of data relating to a moving picture is described in a Hypertext file corresponding to the moving picture, thereby to specify where the moving picture is linked.
Hereinafter, a detailed description is given using FIGS. 10(a)-10(c).
A plurality of information sources are connected to the current Internet serving as an information supply system. Each information source contains, as Hypertext files, home pages relating to various kinds of subjects. Now it is assumed that the terminal unit 704 is connected to an information source having a home page (HP) whose title is xe2x80x9cWorld of Dinosaursxe2x80x9d and this home page HP (Hypertext file screen HTs) is displayed on a display of the terminal unit 704.
In this home page HP, as shown in FIG. 10(a), items of moving pictures relating to dinosaurs are represented by character groups xe2x80x9cscene 1xe2x80x9dxcx9cxe2x80x9cscene 3xe2x80x9d. With respect to objects constituting a moving picture of each item, for example, xe2x80x9cscene 1xe2x80x9d, relevant data relating to objects ob1 and ob2 are represented by character groups xe2x80x9cobject 1xe2x80x9d and xe2x80x9cobject 2xe2x80x9d, respectively. The display areas of these character groups are moving picture specifying areas D1xcx9cD3 and relevant data specifying area D11 and D12.
The displayed content of this home page HP is described in a Hypertext file Hf corresponding to the home page. To be specific, as shown in FIG. 10(b), character groups xe2x80x9cscene 1xe2x80x9d mlxcx9cxe2x80x9cscene 3xe2x80x9d m3 and location data b1xcx9cb3 showing the locations of moving pictures data corresponding to the respective scenes are described in pairs, and character groups xe2x80x9cobject 1xe2x80x9d m11 and xe2x80x9cobject 2xe2x80x9d m12 and locational data b11 and B12 showing the locations of relevant data relating to the respective objects are described in pairs.
When the user moves a mouse pointer MP to the area D1 corresponding to the character group xe2x80x9cscene 1xe2x80x9d on the home page HP and clicks the mouse, a new display screen MPs showing moving picture data linked to the character group xe2x80x9cscene 1xe2x80x9d is opened as a screen of moving picture MI. On the moving picture screen MPs, displayed are two objects ob1 and ob2 and display control buttons CB1xcx9cCB4 for controlling the moving picture display. The user can control display of the moving picture by operating the display control buttons CB1xcx9cCB4 with the mouse pointer MP.
When the user wants to see the moving picture corresponding to the character group xe2x80x9cscene 2xe2x80x9d or xe2x80x9cscene 3xe2x80x9d, the user moves the mouse pointed MP from the screen MPs of the moving picture MI to the area D2 or D3 corresponding to xe2x80x9cscene 2xe2x80x9d or xe2x80x9cscene 3xe2x80x9d on the home page HP and clicks the mouse, whereby a new display screen showing moving picture data linked to xe2x80x9cscene 2xe2x80x9d or xe2x80x9cscene 3xe2x80x9d is opened as a moving picture screen.
Furthermore, when the user wants to retrieve the relevant data of the object ob1 being a constituent of the moving picture MI of xe2x80x9cscene 1xe2x80x9d, the user moves the mouse pointer MP from the moving picture screens MPs to the area D11 corresponding to the character group xe2x80x9cobjects 1xe2x80x9d on the home page HP and clicks the mouse, whereby the relevant data of the object ob1 lined to the xe2x80x9cobject 1xe2x80x9d is displayed.
However, when accessing the moving picture data corresponding to the respective scenes through the Hypertext file HP as mentioned above, it is necessary to move the mouse pointer MP between the screen MPs of the moving picture MI and the Hpertext screen HTs (home page HP). Therefore, at least two screens, i.e., the moving picture screen MPs and the Hypertext screen HTs, must be displayed on the display of the terminal unit 704. Or, display switching between the screen MPs and the screen HTs must be carried out on the display.
Further, locational information showing resource locations of relevant data relating to the object that appears in each scene (for example, when the object is a dinosaur, its figure, habitat and the like) is also described in the Hypertext file. So, when retrieving the relevant data, the user must move the mouse pointer MP from the screen MPs of the moving picture MI to the Hypertext file screen HTs.
Particularly, in the above-mentioned object-by-object coding and decoding method, a picture consists of a plurality of objects, i.e., sub-pictures, and each object has relevant data and, moreover, the relevant data of the respective objects are stored in different servers.
For example, assuming that the relevant data of the objects ob1 and ob2 constituting the moving picture M1 are respectively stored in the servers 706 and 707 shown in FIG. 9, when the user accesses the relevant data corresponding to the object ob1 (ob2), the user must access the relevant data stored in the server 706 (707) from the screen MPs of the moving picture MI showing these objects via the corresponding Hypertext file screen HTs, resulting in poor operability in accessing the relevant data. Furthermore, in this case, it is difficult to manage the locational information of the relevant data corresponding to the respective objects. That is, when one of the objects constituting the moving picture is changed, it is necessary to change the description about locational information of relevant data of the object changed in the Hypertext file.
Furthermore, when information described in the Hypertext file is lost due to transmission error, the locational information of the relevant data of the respective objects constituting the moving picture is lost as well, and the user cannot access the relevant data of the objects using only the moving picture data described in the image file.
It is an object of the present invention to provide a data structure for image transmission and an image transmission method that can improve the operability in accessing relevant data of plural objects constituting an image, that can facilitate management of locational information of the relevant data corresponding to the respective objects, and that can prevent the relevant data of the respective objects from being lost due to transmission error.
It is another object of the present invention to provide a data recording medium that contains a program for implementing, by software, image processing according to the above-mentioned image transmission method or image decoding apparatus.
Other objects and advantages of the invention will become apparent from the detailed description that follows. The detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the scope of the invention will be apparent to those of skill in the art from the detailed description.
According to a first aspect of the invention, there is provided a data structure for coding and transmitting digital image signals corresponding to plural objects constituting a moving picture, wherein a coded image signal obtained by coding the digital image signal corresponding to each object includes display information for image display of the corresponding object, and locational information showing the location of relevant data relating to the corresponding object. Therefore, the user can access relevant data of a desired object by only specifying the object on the display screen of the moving picture. As a result, controllability in accessing relevant data is improved. Further, since the display information of each object and the locational information of relevant data of the object are united, the locational information of relevant data can be managed by managing the display information, whereby management of the locational information of relevant data is facilitated. Further, since the locational information of relevant data of each object is transmitted simultaneously with transmission of the display signal of the object, loss of relevant data due to transmission error is significantly reduced.
According to a second aspect of the invention, there is provided a method of transmitting digital image signals corresponding to plural objects constituting a moving picture, and the method includes the steps of generating a coded image signal by coding the image signal corresponding to each object; and transmitting the coded image signal corresponding to each object after adding locational information showing the location of relevant data of the object to the coded image signal. Therefore, as described above, controllability in accessing relevant data corresponding to each object is improved, and management of the locational information of relevant data is facilitated. Further, loss of relevant data due to transmission error is almost avoided.
According to a third aspect of the invention, there is provided an image decoding apparatus which receives a coded image signal obtained by coding digital image signals corresponding to plural objects constituting a moving picture, the coded image signal including display information for image display of the respective objects and locational information showing locations of relevant data relating to the respective objects, and performs a prescribed decoding process to the coded image signal. The apparatus includes transmission means for transmitting an access signal for accessing the relevant data to an information source where the relevant data is located, according to the locational information of the relevant data; and a decoding unit for decoding the display information. Therefore, as described above, controllability in accessing relevant data corresponding to each object is improved, and management of the locational information of relevant data is facilitated. Further, loss of relevant data due to transmission error is almost avoided.
According to a fourth aspect of the invention, in the image decoding apparatus according to the third aspect, a coded bit stream constituting the coded image signal corresponding to each object has a data structure in which a location bit string constituting the locational information of relevant data is given to every L display bit strings constituting the display information and corresponding to L frames (L: integer, Lxe2x89xa71). Therefore, relevant data that vary according to the motion of the object can be obtained by simple operation on the display screen.
According to a fifth aspect of the invention, in the image decoding apparatus according to the third aspect, a coded bit stream constituting the coded image signal corresponding to each object comprises a plurality of display bit strings constituting the display information corresponding to each object, and a location bit string constituting the locational information of relevant data corresponding to each object, the location bit string being located at the head to the coded bit stream. Therefore, when transmitting the coded image signal, the locational information of relevant data corresponding to the object is transmitted first, so that the locational information is avoided from being lost due to transmission error.
According to a sixth aspect of the invention, there is provided an image decoding apparatus which receives a coded image signal obtained by coding digital image signals corresponding to plural objects constituting a moving picture, the coded image signal including display information for image display of the respective objects and table information comprising object identifiers for identifying the respective objects and locational information showing the location of relevant data relating to the objects, and performs a prescribed decoding process to the coded image signal. The apparatus includes transmission means for transmitting an access signal for accessing the relevant data to an information source where the relevant data is located, according to the locational information of the relevant data; and a decoding unit for decoding the display information. Therefore, locational information of relevant data of all the objects constituting the moving picture can be obtained from the table information.
According to a seventh aspect of the invention, there is provided an image decoding apparatus which receives a coded image signal obtained by coding digital image signals corresponding to plural objects constituting a moving picture, the coded image signal including display information for image display of the respective objects and composition information used for composition of plural objects, and performs a prescribed decoding process to the coded image signal. The apparatus includes transmission means for transmitting an access signal for accessing the relevant data to an information source where the relevant data is located, according to locational information of the relevant data, the locational information being included in the composition information corresponding to each object together with an object identifier for identifying the object; and a decoding unit for decoding the display information. Therefore, relevant data of the respective objects can be obtained according to the composition information.
According to an eighth aspect of the invention, there is provided a data structure for coding and transmitting digital signals corresponding to plural pieces of individual information constituting a single information group, wherein a coded signal obtained by coding the digital signal corresponding to each individual information includes a reproduction signal for reproducing the corresponding individual information, and locational information showing the location of relevant data relating to the corresponding individual information. Therefore, controllability in accessing relevant data corresponding to each object is improved, and management of the locational information of relevant data is facilitated. Further, loss of relevant data due to transmission error is almost avoided.
According to a ninth aspect of the invention, there is provided a data recording medium containing a program for making a computer perform an image signal transmission, wherein the program is constructed so that the computer performs an image signal transmission according to the image transmission method of the second aspect. Therefore, it is possible to realize, by a computer, an image transmission process that improves controllability in accessing relevant data of objects, that facilitates management of locational information of relevant data, and that hardly produces loss of relevant data due to transmission error.
According to a tenth aspect of the invention, there is provided a data recording medium containing a program for making a computer perform an image signal transmission, wherein the program is constructed so that the computer performs an image signal transmission according to the image transmission method of the third aspect. Therefore, it is possible to realize, by a computer, an image decoding process that provides high controllability in accessing relevant data of objects constituting a moving picture.
According to an eleventh aspect of the invention, there is provided a data structure for coding and transmitting digital image signals corresponding to plural objects constituting a moving picture, wherein a coded image signal obtained by coding the digital image signal corresponding to each object includes display information for image display of the corresponding object, and locational information showing the location of the display information. Therefore, on the receiving end, the user can access the information source of display information of each object by receiving at least the locational information of the display information. So, image display of each object can be performed without the display information of the object.
According to a twelfth aspect of the invention, there is provided a method of transmitting digital image signals corresponding to plural objects constituting a moving picture, and the method includes the steps of generating a coded image signal by coding the image signal corresponding to each object; and transmitting the coded image signal corresponding to each object after adding locational information showing the location of display information for displaying the object to the coded image signal. Therefore, on the receiving end, the user can access the information source of display information of each object by receiving at least the locational information of the display information. So, image display of each object can be performed without the display information of the object.
According to a thirteenth aspect of the invention, there is provided an image decoding apparatus which received a coded image signal obtained by coding digital image signals corresponding to plural objects constituting a moving picture, the coded image signal including display information for image display of the respective objects and locational information showing the location of the display information, and performs a prescribed decoding process to the coded image signal. The apparatus includes transmission means for transmitting an access signal for accessing the display information to an information source where the display information is located, according to the locational information of the display information; and a decoding unit for decoding the display information. Therefore, the user can access the information source of display information of a specific object as desired according to the locational information of the display information. As a result, on the receiving end, edition of the moving picture is facilitated.
According to a fourteenth aspect of the invention, in the image decoding apparatus according to the thirteenth aspect, a coded bit stream constituting the coded image signal corresponding to each object has a data structure in which a location bit string constituting the locational information of the display information is given to every L display bit strings constituting the display information and corresponding to L frames (L: integer, Lxe2x89xa71). Therefore, relevant data of display information of the object are received with high reliability.
According to a fifteenth aspect of the invention, in the image decoding apparatus according to the thirteenth aspect, a coded bit stream constituting the coded image signal corresponding to each object comprises a plurality of display bit strings constituting the display information corresponding to each object, and a location bit string constituting the locational information of the display information of each object, which location bit string is located at the head of the coded bit stream. Therefore, when transmitting the coded image signal, the locational information of display information of the object is transmitted first, so that the locational information of display information is avoided from being lost due to transmission error.
According to a sixteenth aspect of the invention, there is provided an image decoding apparatus which receives a coded image signals obtained by coding digital image signals corresponding to plural objects constituting a moving picture, the coded image signal including display information for image display of the respective objects and table information comprising object indentifiers for identifying the respective object and locational information showing the location of the display information, and performs a prescribed decoding process to the coded image signal. The apparatus comprises transmission means for transmitting an access signal for accessing the display information to an information source where the display information is located, according to the locational information of the display information; and a decoding unit for decoding the display information. Therefore, locational information of display information of all the objects constituting the moving picture can be obtained from the table information. On the receiving side, the user can access the information source of display information of each object according to the table information alone to display the moving picture comprising the plural objects.
According to a seventeenth aspect of the invention, there is provided an image decoding apparatus which receives a coded image signal obtained by coding digital image signals corresponding to plural objects constituting a moving picture, the coded image signal including display information for image display of the respective objects and composition information used for composition of plural objects, and performs a prescribed decoding process to the coded image signal. The apparatus comprises transmission means for transmitting an access signal for accessing the display information to an information source where the display information is located, according to locational information showing the location of the display information of each object, the locational information being included in the composition information corresponding to each object together with an object identifier for identifying the object; and a decoding unit for decoding the display information. Therefore, on the receiving end, the user can obtain display information of a desired object from its information source according to the composition information alone. For example, when a coded image signal of a moving picture sent from an artificial satellite is transmitted through a relay center to a local station, locational information of display information of a specific object (commercial) included in composition information of the coded image signal is changed to one corresponding to the local station in the relay center, whereby a moving picture including a commercial suited to the area of the local station can be reproduced in the local station.
According to an eighteenth aspect of the invention, there is provided a data structure for coding and transmitting digital signals corresponding to plural pieces of individual information constituting a single information group, wherein a coded signal obtained by coding the digital signal corresponding to each individual information includes a reproduction signal for reproducing the corresponding individual information, and locational information showing the location of the reproduction signal of the corresponding individual information. Therefore, on the receiving end, the user can access the information source of the reproduced signal of each individual information by receiving at least the locational information of the reproduced signal of the individual information. So, reproduction of each individual information can be performed without the reproduction signal of the individual information.
According to a nineteenth aspect of the invention there is provided a data recording medium containing a program for making a computer perform an image signal transmission, wherein the program is constructed so that the computer performs an image signal transmission according to the image transmission method of the twelfth aspect. Therefore, it is possible to realize, by a computer, an image transmission process that can access the information source of display information of each object by receiving at least the locational information of the display information on the receiving end and that can perform image display of each object without the display information of the object.
According to a twentieth aspect of the invention, there is provided a data recording medium containing a program for making a computer perform an image signal transmission, wherein the program is constructed so that the computer performs an image signal transmission according to the image transmission method of the thirteenth aspect. Therefore, it is possible to realize, by a computer, an image decoding process that can access the information source of display information of a specific object according to the locational information of the display information and that can facilitate edition of the moving picture on the receiving end.